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ARI  Research  Reports  and  Technical  Reports  are  intended  for  sponsors  of 
R&D  tasks  and  for  other  research  and  military  agencies.  Any  findings  ready 
for  implementation  at  the  time  of  publication  are  presented  in  the  last  part 
of  the  Brief.  Upon  completion  of  a  major  phase  of  the  task,  formal  recom¬ 
mendations  for  official  action  normally  are  conveyed  to  appropriate  military 
agencies  by  briefing  or  Disposition  Form. 
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FOREWORD 


The  Training  Technical  Area  of  the  US  Army  Research  Institute  for 
the  Behavioral  and  Social  Sciences  (ARI)  has  actively  pursued  a  program  of 
research  in  support  of  the  systems  engineering  of  training.  A  major  focus 
of  this  research  is  to  develop  the  fundamental  data  and  technology  neces¬ 
sary  to  field  integrated  systems  for  improving  individual  job  performance. 
Such  systems  include  Skill  Qualification  Testing,  job  performance  aids, 
training  courses  in  schools  and  in  the  field,  performance  criteria  and 
management  and  feedback  systems. 

Training  for  the  Skill  Component,  the  written  part  of  Skill  Qualifi¬ 
cation  tests,  tends  to  vary  widely  from  unit  to  unit.  In  addition,  at  the 
time  this  research  was  conducted  (1981)  it  was  on  this  component  of  Skill 
Qualification  Tests  that  soldiers  performed  least  successfully. 

• 

This  report  describes  a  method  for  training  soldiers  scheduled  to 
take  written  Skill  Qualification  Tests.  The  method  employs  diagnostic 
pretests  which  can  be  developed  and  administered  by  battalion  level  train¬ 
ing  personnel.  The  pretest  method  resulted  in  superior  test  performance. 
Results  of  this  research  have  implications  for  individuals  concerned  with 
preparation  for  a  variety  of  written  tests. 


TRAINING  FOR  SKILL  QUALIFICATION  TESTING 


BRIEF 


Requirement: 

.  To  determine  the  effects  of  administering  a  diagnostic  pretest  on 
soldiers'  SQT  performance. 

Procedure: 

A  pretest  entitled  Skill  Qualification  Training  Diagnostic  was  made 
up  from  the  11B1  Infantryman  Soldier's  Manual  and  1980  SQT  Notice. 

After  field  validation,  locally  developed  Scorable  Units  were  intermixed 
with  Scorable  Ihits  drawn  from  the  1980  Skill  Component  for  enlisted 
infantrymen.  Within  each  1980  Skill  Component  Scorable  Unit,  test 
questions  were  scrambled  and  answer  choices  for  each  question  were 
scrambled  to  prevent  test  compromise.  Participants  were  not  informed 
that  1980  Skill  Component  material  was  part  of  the  Skill  Qualification 
Training  Diagnostic.  The  pretest  was  administered  to  infantry  soldiers 
at  Ft.  Myer  and  Ft.  Bragg.  They  received  immediate  feedback  about  pretest 
performance  and  their  units  received  summaries  of  their  troops'  results. 

Findings : 


1.  Scorable  Units  developed  from  the  SQT  Notice  and  Soldier's 
Manual  closely  resembled  Scorable  Units  in  the  1980  Skill  Component. 

2.  Soldiers  performed  at  about  the  same  level  on  locally-developed 
material  as  on  1980  Skill  Component  material. 

3.  Pretesting  permitted  trainers  to  distinguish  tasks  that  needed 
concentrated  training  from  tasks  on  which  soldiers  were  already  proficient. 

4.  Pretested  soldiers  performed  better  on  the  official  1980  Skill 
Component  than  did  non-pretested  soldiers. 

Utilization  of  Findings: 

Developing  and  administering  pretests  to  troops  scheduled  to  take 
Skill  Qualification  Tests  could  save  training  time  spent  on  tasks  soldiers 
already  know,  could  supplant  post  SQT  remedial  training,  and  could 
reduce  soldiers’  test  anxiety. 
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TRAINING  FOR  SKILL  QUALIFICATION  TESTING 
INTRODUCTION 

The  Skill  Qualification  Test  (SQT)  program  is  a  relatively  new  one. 

It  was  introduced  in  1977  and  is  still  undergoing  development  and  revi¬ 
sion.  Its  principal  instruments  are  performance-based,  criterion-referenced 
tests  of  tasks  critical  to  soldiers'  duty  positions — tasks  detailed  in 
Soldier's  Manuals.  Test  results  are  intended  to  be  used  to  diagnose 
individual  training  needs,  and  to  direct  training  and  personnel  management. 

SQTs  are  typically  broken  down  into  three  components:  on-the-job 
testing  called  the  Job  Site  Component,  hands-on  testing  called  the 
Hands-on  Component,  and  written  testing  called  the  Skill  Component.  The 
Job  Site  Component  includes  such  tasks  as  marksmanship  and  physical 
fitness.  Supervisors  judge  their  soldiers'  proficiency  after  observing 
performance  during  normal  practice  sessions.  The  success  rate  for 
performance  on  this  component  has  been  remarkable,  with  scores  of  100% 
commonly  found.  The  Hands-on  Component  tests  actual  performance  of 
selected  critical  tasks  such  as  operating  a  field  telephone  or  putting 
on  a  protective  mask.  Success  on  this  component  too  has  been  widespread. 

The  written  portion  of  the  test,  the  Skill  Component,  has  achieved  the 
lowest  success  rate  of  the  three  components.  It  is  with  the  written 
component  of  the  SQT  that  this  research  deals. 

The  Skill  Component  (SC)  is  made  up  of  tasks  that  are  impractical, 
and  sometimes  impossible,  to  test  in  a  hands-on  form.  For  example, 
map  reading  is  a  logical  candidate  for  written  testing  and  so  is  calcu¬ 
lating  the  velocity  of  a  stream.  One  way  to  account  for  the  lower 
success  rate  for  the  SC  may  be  by  examining  the  way  many  soldiers  prepare 
for  it  compared  to  the  way  they  prepare  for  the  Hands-on  Component. 

Sixty  days  prior  to  scheduled  testing,  soldiers  are  provided  with 
an  SQT  Notice  that  informs  them,  in  substantial  detail,  about  the  tasks 
that  will  be  tested  in  all  three  components.  For  the  Hands-on  Compo¬ 
nent,  SQT  Project  Officers  are  responsible  for  setting  up  and  equipping 
test  stations  for  each  task  listed.  Soldiers  who  are  to  be  tested 
practice  the  tasks  at  the  stations  at  which  they  eventually  will  be 
tested.  As  they  complete  each  task,  proctors  tell  them  if  the  perform¬ 
ance  was  successful;  i.e.,  if  they  scored  GO  or  NO  GO.  When  soldiers 
finally  take  this  component  for  record,  they  are  performing  familiar 
tasks  in  a  familiar  environment. 

To  prepare  for  the  SC,  soldiers  are  encouraged  to  study  their 
Soldier's  Manuals.  In  some  units  they  may  also  attend  lectures  about 
material  to  be  tested  or  do  hands-on  practice  for  the  SC.  A  better  way, 
however,  to  prepare  soldiers  to  take  the  written  test  may  be  to  imitate 
the  form  of  preparation  for  the  Hands-on-Component — to  prepare  to  perform 
familiar  tasks  in  a  familiar  environment.  The  suggestion  here  is  that 
the  way  to  prepare  to  rke  a  wrJ  en  test  is  by  taking  a  written  test. 

To  explore  this  hy, .  aesis,  written  pretests  were  administered  to 
basic  infantry  soldiers.  They  were  provided  with  immediate  feedback 
about  test  performance  and  each  participant  was  equipped  with  a  record 
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of  his  performance.  In  addition,  each  unit  was  provided  with  a  summary 
of  their  troops'  results.  The  pretest  administered  was  called  Skill 
Qualification  Training  Diagnostic  (SQTD)  because  it  was  intended  to 
diagnose  training  needs  prior  to  formal  Skill  Qualification  Testing. 

OBJECTIVES 

The  objectives  of  this  research  were  to  test  the  effects  of: 

1.  diagnosing  training  needs  prior  to  SQT  administration 

2.  shifting  emphasis  from  remedial  training  (post  SQT)  to 
pretraining  (pre  SQT) 

3.  reducing  artifactual  error  in  the  testing  process 

4.  developing  test  items  from  SQT  Notices  and  Soldier's 
Manuals . 

METHOD 

Subjects 

More  than  1000  11B  noncommissioned  officers  (NCOs)  and  enlisted  men 
stationed  at  Ft.  Bragg  and  Ft.  Myer  participated  in  this  research.  NCOs 
participated  in  order  to  become  familiar  with  testing  procedures  and  to 
learn  in  which  areas  their  troops  needed  training.  The  data  analysis  in 
this  report  includes  only  those  enlisted  personnel  for  whom  official  SQT 
scores  were  made  available. 

Test  Development 


Using  SQT  Notice  11B  2180(AC)TR1/2N  and  Soldier’s  Manual  FM  7-1131/2 
dated  7  July  1978,  test  items  were  developed  that  covered  all  12  tasks 
to  be  included  in  the  1980  Skill  Component  for  11B1  soldiers.  Every 
effort  was  made  to  duplicate  style,  format,  and  language  common  to  prior 
years'  written  SQTs  for  basic  infantry  soldiers.  Items  developed  were 
used  during  the  validation  phase  of  this  research  and,  after  minor 
revision,  during  the  main  data  gathering  phase. 

Field  SOPs  were  developed  for  both  validation  and  SQTD  construction 
phases  of  the  research  (see  appendix).  Answer  sheets  for  validation 
were  locally  developed  (see  appendix).  Mark  sense  answer  sheets,  the 
same  answer  sheets  used  for  the  official  SQT,  were  obtained  from  the 
Array  Training  Support  Center  for  use  during  the  data  gathering  phase. 

Test  Validation 


Sixty-nine  soldiers  assigned  to  the  11B10  Military  Occupational  Specialty 
(MOS)  participated  in  this  effort.  Test  items  were  typed  individually 
on  5  x  8  index  cards.  In  an  interview  format,  the  items  were  presented  to 
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soldiers  one  at  a  time  and  each  was  asked  to  comment  on  the  content  of 
the  item  and  whether  it  tapped  information  critical  to  the  11B1  MOS.  At 
the  end  of  the  interview,  each  soldier  was  presented  with  a  record  of 
his  performance  (see  appendix)  that  could  be  used  to  direct  his  training 
for  the  SC.  An  item  analysis  was  performed  on  these  data  and  items  were 
modified  in  accordance  with  indicators  that  emerged. 

SQTD  Construction 

The  twelve  Scorable  Units  (groups  of  questions  under  single  task  headings) 
locally  developed  from  the  SQT  Notice  and  Soldier's  Manual  were  intermixed 
with  twelve  Scorable  Units  taken  from  the  1980  Skill  Component.  Besides 
intermixing  the  task  groupings,  1980  Skill  Component  items  under  each 
task  heading  were  scrambled  and  the  answer  choices  following  each  item 
were  scrambled  to  prevent  test  compromise.  Participants  were  not  in¬ 
formed  that  that  SC  questions  were  included  in  the  SQTD.  Official  SC 
questions  were  included  to  permit  a  comparison  of  locally  developed 
questions  wich  official  SC  questions  in  terms  of  effects  on  1980  SC 
performance . 

The  24  Scorable  Units  (12  locally  developed  and  12  official)  were 
separated  into  4  test  booklets  each  of  which  included  half  of  the  tasks 
to  be  tested  in  the  1980  written  SQT.  Booklets  1  and  2,  therefore, 
covered  exactly  the  same  tasks,  but,  generally,  included  different 
questions  on  those  tasks  (in  some  cases,  questions  developed  from  the 
Notice  and  Soldier's  Manual  turned  out  to  be  the  same  questions  included 
in  the  1980  SC).  The  same  was  true  of  booklets  3  and  4.  The  decision 
to  use  only  half  of  the  full  number  of  tasks  for  each  session  was  based 
on  the  desire  to  confine  sessions  to  one  hour  each  (15  min.  introduction, 
30  min.  testing,  15  min.  feedback).  Each  booklet  was  made  up  in  the 
Skill  Component  format.  Each  was  headed  by  a  cover  sheet  that  was 
followed  by  a  sheet  providing  instructions  about  the  wav  to  take  the 
test,  sample  questions,  and  a  sample  answer  sheet. 

The  following  tasks  were  included  in  the  1980  SC  for  Track  1 
infantrymen: 

1  -  Perform  mouth-to-mouth  resuscitation  and  external  heart  massage 

2  -  Apply  first-aid  for  wet  or  cold  injuries 

3  -  Identify  a  chemical  agent  using  ABC-M8  Detector  Paper 

4  -  Administer  antidote  to  blood-agent  casualty 

5  -  Move  as  a  member  of  a  fire  team 

6  -  Select  temporary  battlefield  positions 

7  -  Use  visual  signals  to  control  movement  (dismounted) 

8  -  Identify  terrain  features  (natural  and  manmade)  on  the  map 

9  -  Determine  the  grid  coordinates  of  a  point  on  a  military 

map  using  the  military  grid  reference  system 


3 


10  -  Conduct  a  preoperational  inspection  of  the  Dragon  tracker 

and  round 

11  -  Perforin  immediate  action  procedures  for  a  Dragon  misfire 

12  -  Neutralize  enemy  mines 

In  future  sections,  these  tasks  will  be  identified  by  number. 

RESULTS 

Pretest  Development 

Test  questions  developed  by  using  the  SQT  Notice  and  Soldier's  Manual 
proved  to  be  a  good  match  with  questions  in  the  1980  Skill  Component. 

In  the  case  of  tasks  3,  7,  8  and  9,  description  of  material  to  be  tested 
which  was  offered  in  the  SQT  Notice  was  detailed  enough  to  permit  both 
sets  of  questions  to  be  essentially  identical.  Therefore,  one-third  of 
a  locally  developed  pretest  prepared  without  exposure  to  the  1980  SC 
exactly  matched  the  SC.  Questions  developed  to  pretest  tasks  1,  2,  4, 

5,  and  12  were  similar  to  those  in  the  SC.  That  is.  they  tended  to  tap 
the  same  information  needed  to  answer  SC  questions.  Questions  developed 
for  tasks  6,  10  and  11  were  substantially  different  from  questions  on 
these  tasks  in  the  SC.  It  seems  reasonable  to  assume,  however,  that  if 
soldiers  were  unable  to  answer  questions  on  any  part  of  a  task,  a  thorough 
review  of  the  whole  task  would  be  needed  before  record  testing.  That 
is,  the  assumption  was  made  that  good  or  poor  performance  on  locally 
developed  Scorable  Units  which  differed  from  Units  in  the  SC  would  be  a 
valid  indicator  of  preparation  needs.  Table  1  compares  soldiers'  performance 
on  ARI-developed  questions  with  those  of  the  1980  SC  used  in  the  SOTD. 


Table  1 


Percent  of  Soldiers  Who  Scored  GO  on  Locally 
Developed  and  SC  Questions  on  the  SQTD 


TASK  ARI  N 

No .  %  GO 


SC  N 

%  GO 


z 


Old  Guard 


1 

66 

(99) 

51 

(83) 

+2.05* 

2 

84 

(56) 

87 

(114) 

-  .53 

3 

50 

(114) 

41 

(56) 

+1.09 

4 

4 

(99) 

6 

(83) 

-  .62 

5 

10 

(99) 

22 

(83) 

-2 . 22* 

6 

93 

(83) 

75 

(99) 

+3.27* 

7 

91 

(56) 

94 

(114) 

-  .71 

8 

31 

(99) 

43 

(83) 

-1.69 

9 

48 

(114) 

45 

(56) 

+  .37 

10 

25 

(114) 

34 

(56) 

-1.22 

11 

47 

(83) 

38 

(99) 

+1.23 

12 

32 

(56) 

20 

(114) 

+1.71 

X  =  48 

X  =  46 

SD  =  30 

SD  =  27 

82d  Airborne 

1 

65 

(51) 

62 

(42) 

+  .29 

2 

89 

(63) 

75 

(68) 

+2.09* 

3 

38 

(68) 

40 

(63) 

-  .24 

4 

14 

(42) 

10 

(51) 

+  .60 

5 

26 

(42) 

18 

(51) 

+  .94 

6 

80 

(51) 

88 

(42) 

-1.04 

7 

92 

(63) 

87 

(68) 

+  .94 

8 

48 

(42) 

47 

(51) 

+  .10 

9 

44 

(68) 

29 

(63) 

+1.78 

10 

43 

(68) 

4  6 

(63) 

+  .35 

11 

63 

(51) 

59 

(42) 

-  .40 

12 

6 

(63) 

9 

(68) 

-  .65 

X  =  51 

X  =  47 

SD  =  28 

SD  =  28 

£  < 

.05 

z  =  1 

.96  (2  tailed) 
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The  top  half  of  Table  1  displaying  performance  by  Old  Guard  soldiers 
shows  no  significant  differences  between  locally  developed  and  SC  questions 
for  the  majority  of  tasks  (r  =  .94).  The  lower  half  of  the  Table  showing 
performance  by  82d  Airborne  soldiers  shows  a  similar  correspondence 
between  ARI  and  SC  tasks  (r  =  .97).  However,  scoring  procedures  for  ARI- 
developed  questions  were  calibrated,  on  the  basis  of  Old  Guard  data,  so 
that  performance  on  both  locally  developed  and  official  SC  questions 
would  correspond.  That  is,  scoring  for  82d  Airborne  soldiers  was  relaxed 
for  ARI  questions  that  had  been  shown  to  be  more  difficult  than  SC 
questions  and  made  more  rigorous  for  ARI  questions  that  were  easier. 
Nonetheless,  both  halfs  of  Table  1  show  correlations  close  to  the  same 
values.  The  point  here  is  that  a  pretest  developed  from  information 
gathered  from  a  SQT  Notice  and  Soldier's  Manual  can  predict  performance 
on  a  Skill  Component  with  a  high  degree  of  accuracy. 

Diagnostic  Effects 


The  SQTD  was  developed  to  diagnose  training  needs  prior  to  formal  testing. 
As  Table  1  shows,  performance  by  soldiers  on  locally  developed  test 
items  closely  resembles  performance  on  the  SC  items.  Figure  1  displays 
a  ranking  of  ARI-developed  scorable  units  from  easiest  to  hardest  for 
both  groups  of  infantry  soldiers. 

The  tasks  tend  to  fall  into  3  groups.  The  three  tasks  at  the  top  of  the 
figure  are  those  on  which  both  groups  are  proficient  and  to  which  little 
training  time  need  be  devoted.  The  six  tasks  in  the  center  of  the  graph 
require  an  intermediate  amount  of  training,  and  the  three  tasks  at  the 
bottom  of  the  graph  require  the  greatest  concentration  of  training 
effort.  The  intermingling  of  subgroup  percentages  stresses  the  similarity 
of  performan  .e  by  both  groups  of  soldiers  with  two  exceptions.  82d 
Airborne  soldiers  performed  better  on  tasks  covering  the  medium  antitank 
assault  weapon  called  Dragon  than  did  Old  Guard  soldiers.  This  outcome 
is  not  surprising,  however,  since  Old  Guard  infantry  soldiers  do  not 
work  in  their  MOS  and  are  not  equipped  with  Dragons.  Figure  1  also 
shows  ranges  of  performance  restricted  enough  so  that  subgroups  pretested 
(for  example,  platoon  level  groups)  fall  into  a  ranking  close  to  that  of 
the  mean  performance  values  computed  for  the  cumulated  group.  The 
indication  here  is  that  performance  by  small  numbers  of  soldiers  might 
be  used  effectively  to  predict  the  performance  of  a  larger  group. 

SQT  Effects 

It  was  expected  that  a  substantially  greater  number  of  soldiers  whose 
training  needs  were  diagnosed  by  pretesting  would  perform  better  on  the 
SC  of  the  1980  SQT  than  they  had  on  SQTD.  Table  2  shows  z  values  for 
these  correlated  proportions  (these  computations  deal  with  changes  from 
NO  GO  to  CO  corrected  for  changes  from  GO  to  NO  GO). 

The  expectation  was  met  by  the  great  majority  of  the  groups  being  analyzed. 
In  some  cases,  for  example  tasks  2,  6,  and  7  no  great  improvement  was 
anticipated  since  most  soldiers  had  alreadv  achieved  GO  on  these  tasks. 

In  the  case  of  tasks  3  and  8,  a  greater  percentage  of  soldiers  changed 
from  NO  GO  to  GO  than  from  GO  to  NO  GO  but  these  percentages  are  not 
statistically  significant. 
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1 0 T  Al  SOLDIER  AVERAGE  %  GO 
SMAll  GROUP  AVERAGE  %  GO 
Ott>  GIJAKt)  O 
H?l)ABN  A 


r 


Z  A  Z  A 


Task  No. 

Go  to 

No-Go  No-Go  to 

Go  N 

z  Go 

to  No-Go 

No-Go  to 

Go 

N 

z 

Old 

Guard 

1 

18 

26 

(83) 

-1.20 

11 

28 

(99) 

-2.72* 

2 

2 

11 

(26) 

-2.50* 

3 

10 

(114) 

-1.94* 

3 

17 

36 

(56) 

-2.61* 

9 

34 

(114) 

-3.81* 

4 

1 

42 

(83) 

-6.25* 

0 

52 

(99) 

-7.21* 

5 

4 

51 

(99) 

-6.33* 

5 

52 

(83) 

-6.22* 

6 

5 

6 

(83) 

-  .32 

1 

18 

(99) 

-3.89* 

7 

*  5 

5 

(56) 

— 

3 

3 

(114) 

— 

8 

13 

31 

(99) 

-2.71* 

18 

22 

(83) 

-  .63 

9 

10 

25 

(114) 

-2.53* 

9 

20 

(56) 

-2.04* 

10 

4 

44 

(114) 

-5.77* 

11 

46 

(56) 

-4.63* 

11 

7 

35 

(83) 

-4.32* 

8 

29 

(99) 

-3.45* 

12 

4 

68 

(56) 

-7.55* 

3 

62 

(114) 

-7.32* 

X 

=  7  X  = 

32 

X  =  7 

X  = 

31 

SD 

=  6  SD  = 

19 

SD  =  5 

SD  = 

18 

82  Airborne 

1 

23 

12 

(51) 

+1.86 

2 

29 

(42) 

-4.84* 

O 

11 

6 

(63) 

+1.21 

4 

19 

(68) 

-3.12* 

3 

19 

22 

(68) 

-  .31 

19 

29 

(63) 

-1.44 

4 

9 

33 

(42) 

-3.70* 

0 

37 

(51) 

-6. OS* 

5 

17 

29 

(42) 

-1.77* 

0 

39 

(51) 

-6.24* 

6 

O 

25 

(51) 

-4.42* 

5 

12 

(42) 

-1.70* 

7 

2 

8 

(63) 

-1.89* 

4 

8 

(68) 

-1.16 

8 

17 

26 

(42) 

-1.37 

20 

27 

(51) 

-1.02 

9 

13 

26 

(68) 

-2.08 

6 

43 

(63) 

-5.29* 

10 

10 

31 

(68) 

-3.28* 

9 

32 

(63) 

-4  . S4* 

11 

10 

29 

(51) 

-3.04* 

7 

29 

(42) 

-3.67* 

12 

0 

49 

(63) 

-7.00* 

4 

35 

(68) 

-4.97* 

X 

=  11  X  = 

25 

X  7 

X  = 

28 

SD 

=  7  SD  = 

12 

SD  =  6 

SD  = 

11 

*_£^.05  z  =  1.64  (one  tailed) 
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Another  approach  to  measuring  SQT  effects  is  to  compare  the  percent  of 
pretested  soldiers  who  scored  GO  on  each  task  with  the  percent  of  soldiers 
who  were  not  pretested  but  scored  GO.  Table  3  shows  these  comparisons. 
These  data  represent  performance  by  82d  Airborne  soldiers  only,  since 
the  arrangement  to  pretest  Old  Guard  soldiers  included  an  agreement  to 
completely  pretest  all  infantry  enlisted  soldiers  and  NCOs.  There  is, 
then,  no  non-pretested  Old  Guard  comparison  group.  The  notable  point 
about  these  data  is  that  the  non-pretested  groups  consist  of  soldiers 
assigned  to  the  same  battalions  as  the  pretested  groups.  Insofar  as  the 
SQTD  drove  training  for  all  enlisted  infantrymen  within  each  battalion, 
any  comparative  effects  would  be  a  function  of  SQTD  experience  alone. 

That  is,  they  would  result  from  increased  familiarity  with  the  SQT 
testing  materials  and  procedures  (reduction  of  artifactual  error  in  the 
testing  process),  and  immediate  feedback  about  performance.  In  the 
great  majority  of  cases  more  pretested  soldiers  scored  GO  on  the  SC 
tasks  that  did  non-pretested  soldiers,  and  in  eight  instances  these 
differences  are  significant. 

The  principal  focus  of  this  research  was  to  determine  whether  diagnostic 
pretesting  would  improve  soldiers'  performance  on  written  SQTs. 

Table  4  compares  official  SC  performance  by  SQTD  pretested  Old  Guard 
soldiers  with  performance  by  the  whole  population  of  enlisted  infantrymen. 
Values  for  the  Army  population  are  adjusted  to  exclude  pretested  soldiers. 

With  one  exception,  more  pretested  soldiers  scored  GO  on  the  SC  tasks 
than  did  the  population  of  infantry  soldiers,  and  in  16  instances,  these 
differences  are  statistically  significant.  The  single  exception  is  a 
task  on  which  both  groups  of  pretested  soldiers  were  proficient  prior  to 
SQT  training  on  the  task. 

In  summary,  this  research  demonstrated  that  diagnostic  pretests  developed 
by  trainers  can  predict  soldiers  training  needs  even  if  only  a  sample  of 
soliders  scheduled  to  take  written  SQTs  are  pretested.  Most  importantly, 
it  showed  that  pretesting  can  promote  better  performance  on  written 
SQTs. 


CONCLUSIONS 

The  use  of  diagnostic  pretests  may  be  the  most  efficient  and  effective 
way  to  prepare  soldiers  to  take  written  SQTs.  First,  they  can  allow 
trainers  to  distinguish  tasks  soldiers  already  know  from  those  that  need 
intensive  training  and  so  avoid  squandering  limited  training  time. 
Second,  these  pretests  shift  emphasis  to  individual  training  before 
testing  rather  than  posttest,  remedial  training,  which  may  be  difficult 
to  schedule  because  of  delays  in  receiving  test  results  or  interference 
resulting  from  cycling  to  collective  field  training.  Finally,  if  the 
pretest  is  set  up  to  mimic  the  SC  environment  as  closely  as  possible, 
soldiers  may  acquire  a  greater  feeling  of  mastery  of  the  test  situation 
which  might  result  in  better  performance  on  the  SC. 
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Table  3 


Differences  Between  Pretested  and  Non-Pretested 
Soldiers  Who  Scored  GO  on  the  SC 

ARI  Scorable  Units 


Task  No.  Pretested 


%  GO 

N 

1 

55 

(51) 

2 

87 

(63) 

3 

43 

(68) 

4 

43 

(42) 

5 

38 

(42) 

6 

98 

(51) 

7 

95 

(63) 

8 

59 

(42) 

9 

57 

(68) 

10 

63 

(68) 

11 

84 

(51) 

12 

56 

(63) 

X  =  65 
SD  «  21 


SC  Scorable 


1 

88 

(42) 

2 

87 

(68) 

3 

49 

(63) 

4 

47 

(51) 

5 

51 

(51) 

6 

95 

(42) 

7 

96 

(68) 

8 

53 

(51) 

9 

65 

(63) 

10 

68 

(63) 

11 

83 

(42) 

12 

40 

(68) 

X  -  69 

SD  -  20 

*  p  <  .05 


Not 

Pretested 

GO 

N 

z 

62 

(444) 

-  .97 

79 

(363) 

+1.48 

45 

(363) 

-  .31 

38 

(444) 

+  .64 

25 

(444) 

+1.83* 

89 

(444) 

+2.00* 

93 

(363) 

+  .59 

45 

(444) 

+1.75* 

57 

(363) 

— 

60 

(363) 

— 

73 

(444) 

+1.69* 

23 

(363) 

+5.50* 

X  =  57 
SD  =  23 


Units 


62 

(444) 

+3.38* 

79 

(363) 

+1.54 

45 

(363) 

•  +  .60 

38 

(444) 

+1.29 

25 

(444) 

+3.61* 

89 

(444) 

+1.22 

93 

(363) 

+  .94 

45 

(444) 

+1.08 

57 

(363) 

+1.27 

60 

(363) 

+1.23 

73 

(444) 

+1.41 

23 

(363) 

+2.98* 

X  -  57 
SD  *  23 


2  *  1.64  (one  tailed) 


Table  4 


Differences  Between  Pretested  Old  Guard  Soldiers  and 
Non-Pretested  Army-wide  Infantrymen  Who  Scored  GO  on  the  SC 


Task  Old  Guard  Army-Wide 

No.  7.  GO  (N=56-114)  7.  GO  (N=6910)  z 


API  Scorable  Units 


1 

76 

63 

+2.60* 

2 

93 

82 

+2.20* 

3 

73 

47 

+5 . 20* 

4 

45 

41 

+  .80 

5 

58 

24 

+8.50* 

6 

94 

89 

+1.67* 

7 

91 

93 

-  .67 

8 

53 

37 

+3.20* 

9 

61 

51 

+2.00* 

10 

65 

54 

+2.20* 

11 

76 

59 

+3.40* 

12 

77 

27 

+8.33* 

X  =  72 

X  -  56 

SD  -  16 

SD  =  23 

SC  Scorable  Unit 


1 

71 

61 

+2.00* 

2 

94 

81 

+3.25* 

3 

66 

41 

+2.71* 

4 

58 

40 

+3.60* 

5 

70 

24 

+9.02* 

6 

92 

90 

+  .67 

7 

94 

93 

+  .50 

8 

47 

39 

+1.60 

9 

54 

51 

+1.43 

10 

61 

54 

+1.00 

11 

65 

60 

+1.00 

12 

55 

29 

+6.50* 

X  -  69 

X  -  56 

SD  -  16 

SD  -  23 

z  ■  1.65  (one  tailed) 
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Trainers  who  make  up  diagnostic  pretests  would  have  an  opportunity  to 
review  tasks  less  frequently  practiced  in  their  units.  Also,  they 
would  be  reminded  of  differences  between  local  and  Soldier’s  Manual 
procedures  on  some  tasks.  In  addition,  trainers  can  trade  off  remedial 
training  time  against  time  invested  in  making  up  diagnostic  pretests. 


I 


i 
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APPENDIX 


SOP 


SQTD  (DEVELOPMENTAL) 


A.  Introduce  self 

B.  Identify  Organization 

C.  Describe  purpose  of  Mini-SQT 

D.  Describe  task(s)  to  be  performed 

E.  Emphasize  '[for  research,  not  for  record" 

F.  Fill  out  identifying  data  on  Record  Sheet 

G.  Fill  in  task  identification  on  participant's  take-away  sheet 

H.  Display  cards  one-by-one  and  question  participant  while  filling  out  Record  Sheet: 

"Would  you  please  r_ad  the  General  Situation  and  try  to  answer  the  question  printed  on 
this  card?"  '  INTER'- EWER:  RECORD  ANSWER  BY  LETTER  OR  RECORD  0  IF  PARTICIPANT  CAN'T  ANSW 

1.  "Is  the  General  Situation  clear/confusing?"  RECORD  YES  OR  NO 

2.  "Is  the  question  clear/conf using?"  RECORD  YES  OR  NO 

3.  "Are  the  answers  clear/confusing?"  RECORD  YES  OR  NO 

4.  IF  PARTICIPANT  IS  UNABLE  TO  ANSWER  QUESTION,  ASK  "why  not?"  DESCRIBE  BRIEFLY 

5.  IF  PARTICIPANT  ANSWERED  QUESTION,  ASK  "How  sure  are  you  that  the  answer  you 
chose  is  the  right  one?" 

a.  I'm  completely  sure  that's  the  answer 

b.  I'm  pretty  sure 

c.  I'm  not  really  sure 

d.  I  just  guessed 

RECORD  PARTICIPANT’S  CHOICE 

6.  "In  your  opinion,  is  the  material  that  this  question  is  based  on  CRITICAL  to  your 
MOS?  That  is,  do  you  think  it's  a  question  that  should  be  included  in  your  test?" 

RECORD  YES’  OR  NO 

RECORD  ANY  RELEVANT  COMMENTS 

I.  Provide  or  confirm  correct  answer  (back  of  each  card,  lower  right  corner).  Review 
question  if  necessary. 

J.  Check  appropriate  response  category  on  participant's  take-away  sheet. 

K.  After  6  Scorable  Units  are  surveyed,  explain  information  in  take-away  sheet  to 
participant . 

L.  Thank  participant. 
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Third,  also  try  not  to  skip  any  questions.  Fill  in  an  answer  for  each 
question  as  you  come  to  it.  For  just  about  everyone  taking  the  test,  there  will 
be  more  than  enough  time  to  review  all  of  the  questions. 

Fourth,  there  is  one  and  only  one  correct  answer  for  each  question,  you’ll 
know  you  made  an  error  if  you  see  that  you  filled  in  more  than  one  circle  in  any 
row. 


Finally,  all  of  the  questions  in  the  test  booklet  are  numbered  consecutively, 
so  that  each  question  number  exactly  corresponds  to  the  same  number  on  the 
answer  sheet. 

Today  is  your  opportunity  to  learn  how  to  fill  out  the  answer  sheet  correctly. 

If  you  have  any  questions  about  how  to  fill  it  out,  don't  hesitate  to  ask.  Just 
signal  one  of  the  people  administering  or  proctoring  the  exercise. 

After  you  have  had  30  minutes  to  take  this  exercise,  we'll  distribute 
Results  Sheets  like  this. 

DISPLAY  RESULTS  SHEET 

On  these  sheets  you  can  mark  which  questions  you  answered  correctly  or  incorrectly. 
Take  the  marked  up  sheet  with  you  when  you  leave  here  today. 

DISTRIBUTE  BLACK  PENCILS 

DISTRIBUTE  ANSWER  SHEETS 

Look  at  the  front  of  your  answer  sheet.  You  can  see  that  it  is  arranged  in 
blocks  with  large  blue  numbers. 

DISPLAY  FRONT  OF  ANSWER  SHEET— POINT  TO  BLOCK  1. 

Look  along  the  top  row  of  block  1.  Print  the  letters  of  your  last  name  in  the 
white  squares  along  the  top  row  starting  all  the  way  over  to  your  left. 

In  the  column  below  each  letter  of  your  last  name,  blacken  the  circle  containing 
that  letter.  For  example,  if  the  name  I  had  written  in  along  the  top  row  were 
SMITH,  I'd  blacken  the  "S"  circle  in  the  first  blue  column,  the  "M"  circle  in 
the  white  column  next  to  it,  the  "I"  circle  in  the  next  blue  column,  and  so 
forth.  If  you  have  any  questions,  please  signal  a  proctor  to  help  you  to  fill 
in  the  answer  sheet. 

WAIT  FOR  LAST  NAMES  TO  BE  ENCODED 

Look  along  the  top  row  of  block  1,  At  the  right  end  fill  in  the  two  white 
squares  with  your  first  and  middle  initials.  Then  blacken  the  corresponding 
circles  in  the  columns  below. 

WAIT  FOR  INITIALS  TO  BE  ENCODED 
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PREVIOUS  PAGE 
IS  BLANK 


Look  at  block  2.  Write  your  social  security  number  in  the  top  row  and  blacken  the 
corresponding  number  circles  underneath. 

WAIT  FOR  SS  NUMBERS  TO  BE  ENCODED 

For  block  number  3,  fill  in  test  number _ in  the  leftmost  white  square  in  the 

top  row  and  blacken  the  corresponding  circle  underneath. 

WAIT  FOR  BLOCK  3  TO  BE  FILLED  IN 

Leave  block  A  blank.  For  block  5,  along  the  top  row  fill  in  80  for  the  year, 
_ for  the  month,  and  _ for  the  day. 

WAIT  FOR  DATE  TO  BE  FILLED  IN 

Leave  block  6  blank.  For  block  7,  write  your  primary  MOS  beside  the  letters  PMOS; 
for  example,  11B10. 

Then  write  your  duty  MOS  beside  the  letters  DMOS.  Even  if  it  is  the  same  as  your 
primary  MOS,  write  it  in. 

At  the  bottom  of  block  7  in  the  space  under  the  letters  P-R-O-M-O  write  a  number 
which  represents  your  total  years  of  formal  education. 

In  block  8,  blacken  the  circle  beside  AA. 

In  block  9,  write  your  pay  grade  in  the  white  blocks  at  the  top  and  blacken  the 
corresponding  circles  underneath. 

For  block  10,  fill  in  your  Unit  Identification  Code  along  the  top  row  and  blacken 
the  circles  in  the  columns  below. 

DISTRIBUTE  TEST  BOOKLETS 

Turn  to  the  booklet  page  with  the  heading  "Skill  Qualification  Training  Diagnostic 
for  MOSC11B10  Kow  to  Take  the  Training  Diagnostic." 

Read  along  with  me: 

This  is  a  multiple-choice  exercise  similar  to  those  you  have  seen  before. 

The  booklet  is  divided  into  UNITS.  There  may  be  as  many  as  8  questions 
in  each  UNIT.  Most  of  the  time  there  will  be  fewer  than  8  questions  in 
a  UNIT. 

Only  one  alternative  is  to  be  selected  for  each  question  in  this  exercise. 

You  are  to  select  the  best  answer  for  each  question  and  blacken  the  circle 
on  the  answer  sheet  that  matches  your  selection.  PLEASE  DO  NOT  MAKE  ANY 
MARKS  IN  THIS  BOOKLET.  Mark  all  answers  on  your  answer  sheet. 
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Look  at  the  sample  questions  and  the  sample  answer  sheet  at  the  bottom  of  the 
page.  Just  above  the  first  question  is  a  General  Situation.  Each  set  of  questions 
covering  a  task,  in  your  Skill  Component  will  be  headed  by  a  General  Situation. 

Sample  question  number  1  is  followed  by  four  answer  choices.  As  you  can  see  from 
the  circle  filled  in  on  row  1  of  the  sample  answer  sheet,  the  correct  answer  is  C. 

Questions  2  through  5  are  set  up  in  a  way  that  it  is  important  to  become  familiar 
with  because  it  is  the  way  some  questions  will  be  set  up  in  your  Skill  Component. 

The  instructions  read  "answer  either  "A"  (TAKE)  or  "B"  (NOT  TAKE)  to  questions  2 
through  5."  You  remember  taking  true/false  tests  in  school.  Well  this  is  very 
much  the  same  thing  except  that  the  Army  is  action  oriented,  so  the  choices 
involve  actions  you  should  take  or  not  take.  The  sample  answer  sheet  shows  that 
question  number  2's  answer  is  B-NOT  TAKE.  Answers  to  questions  3  and  4  are  also 
B-NOT  TAKE  and  question  number  5's  answer  is  A-TAKE,  Do  you  have  any  questions 
about  how  to  choose  an  answer  for  questions  set  up  like  2  through  5? 

ANSWER  ANY  QUESTIONS 

You'll  have  30  minutes  to  work  on  this  exercise.  You  can  turn  your  answer  sheet 
over  to  the  side  marked  SC,  open  your  booklet  to  the  first  question  and  start  now. 

TIME  FOR  30  MINUTES 

COLLECT  BLACK  PENCILS 

DISTRIBUTE  RED  PENCILS 

DISTRIBUTE  RESULTS  SHEETS 

Look  at  the  Results  Sheet  you’ve  just  received.  Notice  that  each  task  that  appeared 
in  the  exercise  just  completed  is  listed  on  the  results  sheet.  Beside  each  task 
is  a  row  of  numbers.  Each  number  stands  for  a  question  on  the  exercise — you’ll 
notice  that  the  numbers  are  consecutive.  I’m  going  to  read  each  question  and  then 
tell  you  which  answer  choice  is  correct.  You’ll  compare  the  choice  you  selected 
for  that  question  number  with  the  one  1  tell  you  Is  correct.  If  they  are  the 
same,  draw  a  circle  around  the  question  number  on  your  Results  Sheet.  If  they  are 
different,  mark  an  X  on  the  question  number  on  your  Results  Sheet.  When  we  finish, 
you'll  be  able  to  tell  from  the  pattern  of  X's  and  circles  where  you  want  to  spend 
more  training  time.  Are  there  any  questions  about  marking  the  results  sheet? 

ANSWER  ANY  QUESTIONS 

READ  ALOUD: 

Task  Title 
Question  Number 
Question 

Correct  answer  and  phonetic  letter  for  that  answer 
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AFTER  THE  CORRECT  ANSWER  FOR  THE  FIRST  QUESTION: 


"if  you  filled  in  (Phonetic  Letter)  on  your  answer  sheet,  draw  a  circle 
around  number  1  on  your  results  sheet.  If  you  filled  in  any  other  letter, 
nark  1  with  an  X" 

COLLECT  BOOKLETS 

COLLECT  ANSWER  SHEETS 

COLLECT  RED  PENCILS 

Keep  in  mind  that  even  if  you  were  very  successful  on  this  exercise,  you  can't  be 
completely  confident  that  you'll  be  equally  successful  on  the  for-record  Skill 
Component.  Only  half  of  the  for-record  Skill  Component  TASKS  were  tested  here, 
many  of  the  questions  are  not  exactly  the  same  questions  you'll  see  in  the  real 
test.  We  want  to  encourage  you  to  be  sure  to  get  an  SQT  Notice  and  study  the 
Soldier's  Manual  tasks  which  your  SQT  Notice  indicates  will  be  tested.  Remember 
that  the  SQT  tests  the  information  in  your  Soldier's  Manual.  In  any  case  in  which 
the  Soldier's  Manual  information  is  different  from  the  way  you  learned  to  perform 
a  task  in  the  field,  answer  SQT  questions  according  to  Soldier’s  Manual  information. 
Another  thing  your  SQT  Notice  will  tell  you  is  how  your  Skill  Component  is  scored. 
Your  score  represents  the  percent  of  Scorable  Units  you  passed — that  is,  on  which 
you  got  GO."  Your  Notice  points  out  that  it  isn't  always  necessary  to  answer  every 
question  about  a  task  correctly  to  score  GO  for  that  Unit.  The  rule  is  that,  for 
Scorable  Units  composed  on  only  1,  or  2,  or  3  questions,  all  questions  must  be 
answered  correctly.  In  the  case  of  more  than  three  questions  in  a  Scorable  Unit, 
however,  the  rule  is  any  3  out  of  4,  any  4  out  of  5,  any  5  out  of  6,  any  5  out  of 
7  and  any  6  cut  of  8  questions  in  a  Scorable  Unit  must  be  answered  correctly  to  be 
scored  GO  on  a  particular  task.  Any  questions? 

ANSWER  ANY  QUESTIONS 

THANK  PARTICIPANTS 
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SKILL  QUALIFICATION  TRAINING  DIAGNOSTIC  01 
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